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Abstract 

In this short note we formulate a infinite-horizon stochastic optimal 
control problem for jump-diffusions of Ito-Levy type as a LP problem 
in a measure space, and prove that the optimal value functions of 
both problems coincide. The main tools are the dual formulation of 
the LP primal problem, which is strongly connected to the notion of 
sub-solution of the partial integro-differential equation of Hamilton- 
Jacobi-Bellman type associated with the optimal control problem, and 
the Krylov regularization method for viscosity solutions. 


1 Introduction 

This short note revisits the infinite-dimensional linear programming (LP) ap¬ 
proach to stochastic optimal control problems. We reformulate the problem 
of minimizing a infinite-horizon cost functional for controlled jump-diffusions 
of Ito-Levy type over a set of admissible controls as a linear program in a 
certain measure space. The linear objective function is the integral of the 
cost function against the occupation measure of the controlled process. The 
main challenge in this LP approach is to prove equality of the optimal value 
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functions of the original control problem V(x) and the associated infinite¬ 
dimensional linear program p(x), and absence of duality gap between the 
primal and dual programs (strong duality). 

Using measure-valued (relaxed) controls, Stockbridge [22] proved the equal¬ 
ity p = V for ergodic optimal control of Markov processes and existence of 
optimal controls. Bhat and Borkar |8j and Kurtz and Stockbridge [T9] ex¬ 
tended these results to the case of feedback controls for time-inhomogeneous 
finite horizon and discounted infinite horizon problems. Cho and Stock- 
bridge mi, Taksar [23] and Helmes and Stockbridge m obtained similar 
results for optimal stopping and singular control problems. 

More recently, using the dual formulation of the primal LP problem and 
viscosity solution theory, Buckdahn et al [TO] proved the equality p = V in 
the case of optimal control diffusions with compact state space. Goreac and 
Serea nu proved the same result for finite-horizon and optimal stopping 
problems. In this paper we show that this approach can be easily extended 
to the jump-diffusion case. We emphasize that our proof does not present 
any significant innovation as we follow closely the arguments in the proof of 
Theorem 6.4 in Jakobsen et al HU. However, to the best of our knowledge, 
this is the first paper that deals with the LP approach to stochastic optimal 
control problems for jump-diffusions. 

Let us briefly describe the contents of this paper. In Section 2 we introduce 
the setting for the optimal control problem of jump-diffusions of Ito-Levy 
type and formulate the primal LP problem associated with the optimal 
control problem and its dual. In Section 3 we recall the definition of viscosity 
solution for partial integro-differential equations and prove the main result 
using the Krylov regularization and results from Jakobsen et al m- 


2 Optimal control problem and LP formulation 

Let (9, F, P) be probability space endowed with a filtration F = {Ft } i>0 sat¬ 
isfying the usual conditions, and let {Wt} t>0 be a standard d —dimensional 
Brownian motion with respect to F. 

Let E = BT \ {0} and let v{dz) be a Levy measure on B(E), that is, a 
non-negative cr-measure satisfying 

/ (|z| 2 A 1) v(dz) < +oo. 

Je 
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Let N ( dz, dt) be a homogeneous Poisson random measure with compen¬ 
sator intensity measure v(dz) dt, and let N(dz, dt) denote the compensated 
Poisson random measure N(dz,dt) := N(dz,dt ) — v(dz) dt. 

Let U be a compact metric space. For each F-adapted [/-valued control 
process u = {ut} t>0 consider the controlled Levy-Ito equation 


dX t = b(X t ,u t )dt + a(X t ,u t )dWt+ [ r)(X t _,u t -, z) N(dz, dt) 

Je 

X 0 = x. 


( 2 . 1 ) 


The coefficients b : R N xU -»• R N , a : R N xU R Nxd and?? : R N xUxE -»■ 
R^ satisfy conditions (1A2I) and (IA3I) below. The class U(x) of admissible 
control policies is defined as the set of control processes u = {ut} t>0 for 
which equation (12.11) has an unique strong solution X x,u = {X^’ u } t>Q . 

Let c > 0 be a fixed discount rate and h : R^ xU — > (—oo, +oo] denote the 
cost-to-go function. Let J be the infinite-horizon discounted cost functional 


J(x,u) := E 



h(X^’ u , lit) dt 


We will use the following norms 


Ho : = SU P 100*01 ; Hi : = \D<j>\ 0 and Hi := Hi + Hi 
and assume the following conditions 


1. The Levy measure u{dz) satisfies 



e m| H v(dz) < oo 


(Al) 


for some m > 0. 

2. There exists K > 0 such that for all u £ U 

|6(-,u)li + |(j(-,u)| 1 + |c(-,u)| 1 + < K 


and 


\v(-,u,z )|i < K 


z \ 1 {o<h|<i}( 2: ) + e^ll{| z |>i}(*) 


(A2) 


(A3) 


3 






Condition (1A1D is equivalent to the Levy process with Levy measure v(dz) 
having finite moments of all orders, see e.g. Applebaum [3, Section 2.5]. 
It is satisfied, for instance, by one-dimensional tempered a-stable processes 
with Levy measure 

Cie~ Xlz Coe~ X2 \ z \ 

"(dz) = 1r+ (z) dz + " 1+ 1r_ (z) dz 

z \z\ 

with C \, C <2 >0, Ai, A 2 >0 and aq, < 2. Under conditions (1A1D - (1A3I) . for 
each u £ U(x) there exists an unique strong solution to equation (12.11) and 
satisfies the following estimate, see e.g. Applebaum [3J Section 6.6] 


E 


sup \X+’ U \ P 
te[o,T] 


<C{ l + \x\ p ) 


( 2 . 2 ) 


for all p > 2. The main object of study of this paper is the stochastic optimal 
control problem 

V(x) := inf J(x,u), x£R N (2.3) 

u&U(x) 

and the following linear programming (LP) formulation: for each x € 
and u £ U(x), denote with j x,u the expected discounted occupation measure 
on B(R n x U ) defined as 


7 X ’ U (Q) ■■= E 


' roo 

/ e - c *i Q (xr, 

Jo 


ut ) dt 


Q £ B{R n x U). 


Using approximation of h by simple functions, it is easy to prove that the 
occupation measure satisfies 


J(x,u)=[ h(y,u)'y x ' u (dy,du). 
Jn N xu 


Let Lp o| (R A7 ) denote the class of C 2 -functions / : R iV —> R with polynomial 
growth. For each u £ U fixed, let A u + J u denote the partial integro- 
differential operator 

A u f(x) := (b(x,u),Df(x)) + ^Tv[(7(x,u)a{x,u)*D 2 f(x)}, 

J u f(x ) := [ {f(x + rj(x,u, z)) - f(x) - 1{| 2 |<i} (rj{x,u, z), Df(x))} u{dz). 
Je 

for / £ Lp o] (R' V ). Here Df(x) and D 2 f(x) denote the vector and square 
matrix of first and second-order partial derivatives of / respectively. 
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Notice that the integral term in the operator J u is well-defined due to the 
exponential decay of the Levy measure v{dz) at infinity (see Assumption 
A.l) and the fact that the singularity at z = 0 is integrable for any / £ 
C 2 (lR, Ar ), see e.g. Applebaum [3J Section 3.3]. 

Using Ito’s formula for Levy-Ito processes, Kunita’s inequality and estimate 
(12.21) . for any / £ C pol (HP V ) and T > 0, we have 


E[e~ cT f(X x ’ u )]-f(x) = E 


/ 


e~ ct {[(A + J)f}(xr, ut) - cf(xr)} dt 


Also from estimate (12.21) we have 


lirn E[e -cT 

T-> oo 


f(X*’ u )\ = 0. 


By the dominated convergence theorem, taking the limit as T —>■ oo it follows 


E 


roo 

/ e- ct [cf-(A + J)f](X?' u ,u t )dt 
Jo 


f(x). 


We have proved that the occupation measure r y x,u satisfies the linear con¬ 
straint, 



+ J u )f](y)^’ u (dy,du) 


f(x), VfeC 2 pol (R N ). (2.4) 


This suggests to consider the following LP problem over the vector space 
A4b(® iV x U) of finite signed measures on B(H N x U) 


p{x) :=inf / h(y , u) p(dy , du) 

Jr n xU 

subject to fi £ A4b(E iV x [/), y > 0 

and [ [cf - (A u + J u )f](y) y(dy,du) = f(x), V/ £ 

Jr n xu 

(2.5) 

Clearly, we have p < V. The main purpose of this note is to prove that in 
fact equality p = V holds. 

In order to formulate a LP problem with the linear constraint (|2.4D , and its 
dual, we recall briefly some facts and notation concerning infinite-dimensional 
linear programming. Two topological real vector spaces X , y are said to 
form a dual pair if there exists a bilinear form (•, •) : X such that 
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the mappings X 3 x —» (x, y) £ 1R for y £ T separate points of X and the 
mappings y 3 y —>• (x,y) G R for x G X separate points of y. 

We endow X with the weak topology a(X,y), i.e. the coarsest topology 
for which the maps X 3 x —> (x, y) G It are continuous for all y G 3- For any 
vector subspace F C X, let F* denote the vector space of linear functionals 
on F with respect to the inherited weak topology. Notice that X* = y 
according to this notation. 

Let X + be a positive cone in X, that is, a non-empty convex cone with 
vertex 0. This induces the vector order 


Xi > X2 iff X\ — X2 G X + . 

The positive cone y + in y is defined as the negative polar 
3 ;+ := {y G y : (x, y) > 0 for all x G X + }. 

Notice that y + is closed, and the identity 

X + = {x G X : (x, y) > 0 for all y G 3 ;+ } 
holds only if X + is closed. 

Let (Z, W) be another dual pair and let C : X —> Z be a continuous linear 
operator. We define the adjoint map C* : W —> T via the relation 

(x,C*w) := (Cx,w) , x G X, w G W. 

Let b £ Z,c £ y be given, and consider the primal linear problem 

( 7 r) minimize (x, c) 

subject to Cx = b, x > 0. 

The (algebraic) dual problem of (P) is given by 

( 7 r*) maximize (b,w) 

subject to — C*w + c > 0, w GW. 

It can be proved that the inequality sup(7r*) < inf (77) holds, see e.g. An¬ 
derson and Nash [2j. If this inequality, usually referred to as weak duality , 
is strict, then we say that there is a duality gap between the primal linear 
program and its dual. 
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We now follow the above definitions (see also mm) to formulate the 
dual problem to (I2.5[) . We set X = .M^E^ x U) and y = Cb(^ W x U), 
the vector space of bounded continuous functions on ~R N x U. These spaces 
form a dual pair with the bilinear form 

</,/*>:= [ /(y > u ) p(dy, du), f G C h (R N X U), p G VWbC^ X U). 

JR N xU 

Finally, let A + J : D(A + J) = Cp 0 i(E' V ) — > B(R N x U) denote the linear 
operator defined by 

[(A + J)f](x,u) := (A u f + J u f)(x), (x,u) G E N x U. 

We take W := D{A + J) = C^R^) and Z := D(A + J)*. We define the 
operator C : X — > Z via its adjoint operator C* : W — > y as follows 

C*f:=cf-(A + J)f, feW. 

With this notation, the linear program (12. 5 p associated with the control 
problem (12.31) now reads 

pfx) =\nt (h,p) 

subject to /i G (E/ v x U) 

T / / — 5 X . 


The dual linear program is given by 

p*(x) = sup {f,S x ) 

subject to / G Cp 0 i(E ,v ) 

-C*f + h>0 

and the weak duality p*(x) < p(x) holds, for all x G E^. Notice that the 
restriction — C* f + h > 0 on / G Cp,,i(E A? ) is equivalent to 

cf(y) ~ h(y,u) - (A +J)(y,u) < 0, Vu, G U, y G E w 

that is, / is a (smooth) sub-solution with polynomial growth of the ellip¬ 
tic partial integro-differential equation (PIDE) of Hamilton-Jacobi-Bellman 
type 

sup {cV(y)-h(y,u)-[(A+ J)V](y)} = 0, x G E^. (HJB) 

u&U 
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Recall that equation (1HJBI) is the equation associated with the dynamic 
programming approach to existence of optimal feedback controls for problem 
m- Using this equation, we can rewrite the dual problem as 


p*(x ) = sup { f(x ) : / £ D(A + J) is sub-solution of equation (1HJBI) } . 
The following is the main result of this paper 

Theorem 2.1. The value function of the control problem H2.3\) satisfies 
V(x) < p*(x ) for all x £ IR/ V . 

As an immediate consequence, we have V = p (optimal values of the LP 
problem (12.51) and the control problem (12.31) coincide) and p = p* (absence 
of duality gap). 


3 Viscosity solutions and Krylov regularization 

Notice that equation (1HJBD can be rewritten in the form 

H(x, V(x), DV[x), D 2 V(x), V(-)) = 0, x £ R N (P) 

where for any (x,r,p, X) £ R iV x K, x x Sat and sufficiently regular 
function V, the nonlinear Hamiltonian H is defined by 


H(x,r,p,X,V(-)) 

= supj cr — h(x,u) — (b(x,u),p) — ^ Tr[cr(x, u)a(x, u)*X] — J u V(x )|. 
u&u 1 2 J 

Because of the degeneracy in the second order differential operator and the 
non-local integral operator J u , elliptic equation ((P|) is expected to have only 
solutions in the viscosity sense. The notion of viscosity solution was intro¬ 
duced by P.L. Lions and M.G. Crandall in the 1980s for first-order Hamilton- 
Jacobi equations. This was later generalized to second-order equations and 
more recently to equations with integro-differential operators. The viscos¬ 
ity solution approach provides an appropriate framework to formulate HJB 
equations for functions that are only assumed to be locally bounded, see e.g. 
Alvarez [1], Bardi |4], Barles et al [5], Barles and Imbert [6], Crandall et al 
[12] . Fleming and Soner m, and the references therein. 
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Definition 3.1. (i) A locally bounded upper semi-continuous (resp. lower 

semi-continuous) function V : M. N —> ft is a viscosity sub-solution 
(resp. super-solution ) of (0 iff for every x £ E; v and </> £ 
such that 

V — cf> attains a global maximum (resp. minimum) at x 


then 

H(x, V(x), Dcffx), D 2 (J)(x), (/>(•)) < 0 (resp. >0) 

(ii) V is a viscosity solution of (P) if it is both sub- and super-solution. 

We follow the regularization method of N.V. Krylov j d8] , originally intro¬ 
duced to estimate the convergence rate of finite-difference approximations 
of equation m in the diffusion case. It has been systematically improved 
by Barles and Jakobsen [7], and Jakobsen et al m among others, to ob¬ 
tain monotone approximation schemes for elliptic and parabolic equations 
of Bellman-Isaacs type. 

For e £ (0,1), consider the auxiliary perturbed equation 

sup H(x + e,V £ (x),DV £ (x),D 2 V £ (x),V £ (-)) = 0, x£R N . (P e ) 

|e|<e 

Notice that this is the HJB equation associated with the control problem 
with control set U x 11(0, e), cost function h(x,u,e) := h(x + e,u) and 
coefficients b, a and r] defined analogously. Let V £ denote the optimal value 
function 

" nOO 

V £ (x) = inf E / e~ ct h(Xf ,u ’ e + et,ut) 

(«,e)e W{x) [Jo 

where U £ (x) is the set of admissible control policies and, for each (u,e) £ 
U £ {x), X x ’ u ’ e denotes the corresponding controlled process. Notice that U{x) 
can be seen as subset of U £ (x) by identifying u £ fl(x) with (u, 0) £ U £ {x). 

Lemma 3.2. V (resp. V £ ) is the unique viscosity solution to equation fRl 
(resp. (P^) in C°’ 1 (IEl JV ) and, for each x £ we have V £ (x) —>■ V[x) as 
£ —>• 0 . 

Proof. Viscosity solution property of the value functions V and V £ is a 
standard well-known result that follows from the dynamic programming 
principle, see e.g. Pham m Section 3], 0ksendal and Sulem [20[ Section 


x £ R 


N 
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10.3] or Bouchard and Touzi [9] Section 5.2], The second assertion follows 
from the continuous dependence result in Theorem 6.2 in Jakobsen et al 

pea- □ 


We also need the following result. For the proof see Lemma 6.3 in Jakobsen 
et al [17] . 

Lemma 3.3. Convex combination of viscosity sub-solutions of (P) is also 
a viscosity sub-solution of (P). 

Proof of Theorem \2.1[ The proof is largely based on the proof of Theorem 
6.4 in m- Using the change of variable y = x + e, the function V £ (■ — e) 
satisfies 


H(y, V £ (y - e), DV £ (y - e), D 2 V £ (y - e), V(- - e)) < 0 


in the viscosity sense, for every |e| < e. That is, V e (- — e ) is viscosity sub¬ 
solution of ((P]) for every |e| < e. Now, for k, e £ (0,1), let V £>K := V e * <f K 
where 

Mx) := 

and 4> is a smooth positive function with unit mass and support in B( 0,1). 
For h < k and a £ hh N define 


Qh(a) ■= a + 


h h 

2’2 


N 



4>K,(y) dy 


and 

V £ , K ,hi x ) : = ^ y £ ix - a)<j> Kth (a). 

a£hZ N 

By properties of modifiers, V £jKt h —► as h — > 0. Moreover, (j> Ki .h.(a) > 0 

only for finitely many a £ h^ N since <p K has compact support, and 


y ] 0k,/). 

a&hZ N 



4>n{y) dy = i- 


By Lemma [3~3l V £>K ^ is a viscosity subsolution of (P), and so is its pointwise 
limit V £jK = lirn/^o V £ ^ ) h by stability results for viscosity subsolutions, see 
e.g. Barles and Irnbert |5J Section 3]. By the definition of the dual problem, 
we have V £)K < p*(x) for all x £ IR: V . The desired result follows by taking 
the limit as e, k —> 0 in conjunction with Lemma 13.21 □ 
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